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1.  INTRODUCTION 

Consider  a system  with  m components.  Let  >>,-  (/  = 1,  2, . be  an  indicator  of  the  state 
of  component  i (i.e.  y,-  = 1 if  component  / is  good  and  = 0 if  component  / fails).  Similarly 
let  y be  an  indicator  of  the  state  of  the  system.  The  boolean  tran.»mission  function 


relates  the  performance  of  the  system  to  the  performance  of  its  components,  and  is  called  the 
“structure  function”. 

The  systems  considered  in  this  note  are  coherent  systems  whose  components  operate 
independently  and  whose  structure  function  is  known. 

The  reliability  of  the  system  is 


p = Pr{y=  1}. 


Some  examples  of  types  of  systems  are: 

(i)  Series  system 

A series  system  is  good  only  if  all  its  components  are  good.  It  has  structure  function 


fiy^yi >’m)=  n >■;. 


(ii)  Parallel  system 

A parallel  system  is  good  if  any  one  of  its  components  are  good.  It  has  structure 
function 


/(>’i.>'2 J’-n)  = 1 - n (I  -y'i). 

i = l 

(iii)  "k  out  of  m"  system 

A "k  out  of  m"  system  is  good  if  any  k of  its  m components  are  good.  The  struc- 
ture function  cannot  be  expressed  in  simple  form. 


In  practice,  systems  are  often  composed  of  blocks  where  the  components  within  a block 
are  linked  together  by  an  arrangement  such  as  one  of  those  given  above,  and  the  blocks  themselves 


are  linked  together  by  some  other  arrangement.  Lloyd  and  Lipow'  (p.  255)  cite  the  case  of  a 
space  vehicle's  temperature  control  system  which  consists  of  a number  of  “k  out  of  m" 
subsystems  linked  together  in  a series — parallel  arrangement. 

It  is  often  too  expensive  to  test  the  system  a number  of  times  to  determine  its  reliability. 
What  can  be  done  however,  is  to  test  the  components  individually.  This  note  is  concerned  with 
the  problem  of  making  inferences  about  system  reliability  on  the  basis  of  pass-fail  data  obtained 
from  tests  on  the  components.  Inferences  generally  fall  into  two  classes,  point  estimates  and 
confidence  limits.  As  pointed  out  by  Easterling^,  the  construction  of  point  estimates  is  relatively 
straightforward.  If  the  results  for  component  i are  at,-  successes  in  «,■  tests  (/  = I,  2, . . . , m)  an 
estimate  of  the  reliability  of  component  / is 

Pi  = xj/ti 

and  the  maximum  likelihood  estimate  of  the  system  reliability/?  is 

/5  = //(^,,  Pi ^m) 

where  h{p,,p2 pm)  is  a function  giving  system  reliability  in  terms  of  the  component 

reliabilities. 

Construction  of  confidence  limits  is  by  no  means  so  straightforward.  The  case  that  has 
received  the  most  attention  is  the  case  of  a series  system.  Exact  methods  have  been  developed, 
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notably  by  Buehler^,  but  these  are  generally  too  cumbersome  computationally,  especially  when 
the  number  of  components  is  greater  than  two  and  the  sample  sizes  (m)  are  not  all  equal.  A 
number  of  approximate  methods  have  been  proposed  to  overcome  this  difficulty.  In  our  opinion 
the  test  of  these  is  the  one  due  to  Easterling^.  A new  approximate  method  is  proposed  in  this 
note  and  its  performance  evaluated  in  certain  special  cases.  The  indications  are  that  this  method 
is  comparable  in  overall  performance  to  the  method  of  Easterling^,  and  has  advantages  in  the 
case  when  no  failures  are  observed  on  the  component  with  the  smallest  sample  size. 

The  remarks  above  refer  to  fixed  sample  size  testing  of  the  components,  i.e.  cases  where 
the  sample  sizes  are  fixed  in  advance  and  do  not  depend  on  the  results  obtained  in  the  tests. 
An  alternative  approach  was  suggested  by  Winterbottom  and  Verrall*.  They  gave  a sequential 
testing  method,  applicable  to  series  or  parallel  systems,  that  minimizes  the  number  of  component 
tests  needed  and  produces  results  in  a form  that  enables  easy  construction  of  confidence  limits. 
This  method  was  generalized  by  Preston’  to  deal  with  any  system  whose  structure  function  is 
known. 


2.  “EXACT”  BINOMIAL  CONFIDENCE  LIMITS 

In  this  section  the  theory  of  “exact”  binomial  confidence  limits  is  outlined;  for  a fuller 
development  the  reader  is  referred  to  Mood  and  Graybill*  or  Kendall  and  Stuart^. 

The  binomial  probability  function  with  sample  size  n and  probability  p is  denoted  by 
B(n,  p)  and  is  given  by 

Pr  {X  = x]  = p^  (I  - p)”-^. 

A lower  confidence  limit  for  p at  a level  of  confidence  a based  on  an  observation  x is  denoted 
by  L^(x).  The  conditions  that  a set  of  numbers  LJix)  (x  = 0,  1, . . . , n)  must  obey  in  order  to 
constitute  a reasonable  set  of  lower  confidence  limits  are: 


Pr  {Lj,X)  ^ p} (X  for  all  p in  (0,  I),  (1) 

ifx,  <.r2  L3,{.r,)  ^ (2) 

and  Li(.v)  should  be  as  large  as  possible  (3). 


Condition  (I)  guarantees  that  the  limits  do  achieve  the  intended  confidence;  condition  (2)  is 
called  the  regularity  condition  and  ensures  that  the  greater  the  observed  value  of  .v,  the  greater 
the  confidence  limit;  and  condition  (3)  states  that,  subject  to  conditions  (I)  and  (2),  the  limits 
should  be  as  large  as  possible  (after  all  Ljl.x)  = 0 for  all  x obeys  conditions  (I)  and  (2)). 

The  limits  generated  by  these  conditions  are  given  by  the  solutions  of  the  equation 


D 

2 (U-V)]  ^ [1  - = 1 - a 

i • * 


(4) 


for  .V  = 1,2 n 


where  L^(o)  is  defined  to  equal  0.  An  alternative  to  solving  equation  (1)  for  LJx)  is  to  use  the 
identity 

n 

^ I4(.V)P  [I  - LJx)]"-^  = /(U.X),  x,n-x  ■ 1 ) (5) 

i « 

where  /(.v,  a,  p)  is  the  incomplete  beta  function 


Us,  *,  /3)  = 

which  is  tabulated  in  Pearson'. 


I’(*  • 
!'(*)  nP) 


rf  'df 
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The  limits  generated  by  (4)  are  referred  to  as  “exact”  because  they  obey  condition  (I). 
The  proof  of  this  proposition  is  given  by  Lloyd  and  Lipow'  p.  209.  They  are  exact  in  the  sense 
that  they  actually  attain  the  confidence  a,  in  contrast  to  such  approximate  limits  as  those  based 
on  the  normal  approximation  to  the  binomial  (see  Hogg  and  Craig*  p.  196)  which  are  not 
guaranteed  to  do  so.  They  are  not  exact  in  the  sense,  used  by  some  authors,  that  they  obey 

Pr{L^(X)  ^ p}  = ec  (6) 

for  all  p.  It  is  only  possible  to  construct  a set  of  intervals  that  are  exact  in  this  sense  by  a 
post-experiment  randomization  procedure  {see  Lloyd  and  Lipow  p.  212)  but  this  procedure 
is  not  widely  used  in  practice. 

Looking  into  this  point  a little  further.  Table  1 shows  the  values  of  Lj(.v)  for  .v  = 0,  1 n 

when  « = 10  and  a = .9,  and  Figure  I shows  the  achieved  confidence  level  a*(p)  where 

a*(p)  = Pr{UX)  ^ p}, 

as  a function  of  p.  it  can  be  seen  that  **(/7)  is  equal  to  <x  for  those  values  of  p For  which 

PJx)  = p,  X — 1,2,...,  10 

and  is  greater  than  a for  all  other  values  of  p.  It  can  also  be  seen  that  LJ,x)  constitute  the  greatest 
set  of  numbers  that  will  achieve  confidence  a;  if  any  one  of  them  was  increased  the  confidence 
would  go  below  a for  some  p.  For  instance,  if  L.q(9)  was  increased  from  0-6632  to  0-68  the 
achieved  confidence  level  (a*(pf)  would  be  lower  than  0-9  for  p in  the  interval  (0-6632,  0-68) 
(see  the  dotted  curve  in  Figure  1). 

It  is  unfortunate  that  the  achieved  level  is  so  much  higher  than  the  intended  level  for  most 
values  of  p.  This  phenomenon  is  a consequence  of  the  fact  that  the  binomial  is  a discrete  dis- 
tribution. It  does  not  occur  with  continuous  distributions.  In  the  next  section  some  approximate 
methods  for  finding  confidence  limits  from  binomial  data  on  the  components  of  a series  system 
will  be  discussed.  One  criterion  for  evaluating  these  methods  that  will  be  used  is  the  comparison 
of  the  intended  and  achieved  confidence  levels.  To  set  the  scene  for  this  sort  of  comparison,  in 
the  remainder  of  this  section  we  will  look  at  the  confidence  level  achieved  by  one  of  the  commonly 
used  approximate  methods  for  constructing  binomial  confidence  limits  (see  Hogg  and  Craig* 
p.  196). 

The  binomial  distribution  B(n.  p)  has  mean  np  and  variance  npq.  The  binomial  can  be 
approximated  by  the  normal  distribution  by  equating  the  mean  and  variance.  Thus  X — np 

y/npq 

is  approximately  normal  with  mean  0 and  variance  1.  An  estimator  of  the  variance  is 


X — np  is  asymptotically  normal  with  mean 

and  it  can  be  shown  that 

/ X 
>n 

V « 


0 and  variance  I.  Then,  from  tables  of  the  normal  distribution. 


is  an  approximate  0-9  confidence  limit  for  p.  Table  I shows  the  exact  (/..g(.v))  and  approximate 
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(/4.9(jf))  limits  when  n = 10,  and  Figure  2 shows  the  confidence  levels  achieved  by  these 
approximate  limits. 

It  should  be  noted  that  this  demonstration  is  not  intended  to  denigrate  the  method  of 
constructing  confidence  limits  based  on  the  normal  approximation.  The  theory  on  which  it  is 
based  is  asymptotic,  and  it  is  well  known  that  it  .should  only  be  used  when  n is  reasonably  large, 
say  30  (see  Crow,  Davis  and  Maxfield*®  p.  51).  The  demonstration  is  intended  to  be  a simple 
example  of  the  sort  of  comparisons  we  shall  be  making  in  later  sections. 

3.  EXACT  CONFIDENCE  LIMITS  FOR  SYSTEM  RELIABILITY 

Consider  a system  with  m components  linked  in  series.  If  the  reliability  (probability  of 
successful  operation)  of  the  /th  component  is  p^,  the  system  reliability  p is  given  by 

m 

p = n Pi. 


Suppose  further  that  binomial  testing  has  been  carried  out  on  each  component  giving  .r,-  successes 
from  Hi  tests  on  component  / (i  = 1,2,.. .,  m).  The  problem  we  address  is  the  determination 
of  a lower  confidence  limit  for  p on  the  basis  of  these  data.  The  theory  of  exact  intervals  as  given 
by  Buehler^  will  be  discussed  in  this  section,  and  some  of  the  better  approximate  methods  will 
be  reviewed  in  the  next  section. 

A lower  confidence  limit  for  p based  on  the  data  x = (x,,  .Vj, . . .,  Xm),  denoted  by  Lj(x), 
must  obey  a similar  set  of  conditions  to  those  given  in  the  previous  section  for  the  case  where 
only  one  item  has  been  tested.  Certainly  it  should  obey 


Pr  {La(x)  < p}  > a 


and  also  L^(x)  should  be  as  large  as  possible, 


(7) 

'8) 


however,  the  appropriate  regularity  condition  poses  something  of  a problem.  For  one  component 
the  regularity  condition  is  straightforward,  it  amounts  to  ordering  the  possible  outcomes  of 
the  experiment  in  terms  of  extremeness.  Certainly  the  result  x = n (n  successes  in  n tests)  is 
more  extreme  than  the  result  x = n — 1 , and  .r  = n — 1 is  more  extreme  than  x = n — 2,  and 
so  on;  thus  one  requires  L^(n)  to  be  greater  than  L^{n  — I),  and  L^(n  — 1)  to  be  greater  than 
L^(n  — 2).  But  with  more  than  one  component  it  is  sometimes  difficult  to  decide  if  one  result 
is  more  extreme  than  another.  For  instance,  with  two  components  each  tested  10  times,  is  the 
result  .X,  = 9,  Xj  = 7 more  or  less  extre  le  than  the  result  x,  = 8,  X2  = 8? 

The  first  step  in  constructing  a set  of  limits  L^{\)  for  all  x is  to  define  an  order  for  the  vectors 
X.  Let  us  suppose  for  the  moment  that  this  has  been  done  (three  criteria  for  ordering  that  have 
been  suggested  will  be  discussed  later)  and  denote  the  order  by  x/,  y = 1,2,..  .,  N,  where 

m 

A = n {n,  -i  1). 

i = I 

The  regularity  condition  implied  by  this  order  is 


L^iXj)  < if  J < k. 


(9) 


Given  the  conditions  (7),  (8)  and  (9)  the  set  of  largest  limits  are  given  by 


L,(X;)  = infjp  ' i:  Bk  = \ -«} 

k = j 


(10) 


where  fl*  = II  ( ' )p/'*(l  — P,)”' 

i I v’f/A-/ 

and  X,  = (X|/,  .Vj; x„,). 

It  can  be  seen  that,  in  the  case  of  one  component  (wi  = I)  equation  (10)  gives  the  same  solution 
for  LJ.x)  as  equation  (4),  by  noting  that  the  term 
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N 

S Bk 

k - i 

reduces  exactly  to  the  lefthand  side  of  equation  (4).  For  a fuller  explanation  of  the  derivation 
of  equation  (10)  the  reader  is  referred  to  Buehler^  and  Winterbottom". 

There  are  two  difficulties  that  arise  when  using  this  exact  method  in  practice,  one  is  defining 
the  order  Xj,  and  the  other  is  solving  equation  (10).  The  latter  problem  can  be  illustrated  with 
an  example.  Suppose  m = 2,  n,  = 10,  = 7,  = 88,  and  the  first  part  of  the  order  has  been 

defined  to  be: 


*88 

= (10,7) 

*87 

= (9,  7) 

*86 

= (10,6) 

*8  5 

= (8,  7) 

*84 

= (9,  6) 

*83 

= (7,  7) 

*82 

= (8,  6), 

and  that  we  wish  to  find  LJ^Xg^),  i.e.  the  lower  confidence  limit  for  p when  9 successes  out  of 
10  tests  were  obtained  on  component  1 and  6 successes  out  of  7 tests  were  obtained  on  compo- 
nent 2.  Equation  (10)  gives 

88 

f-afXsJ  = inf{/’ 1 S fit  = 1 — a} 

k - 84 


where  p — PiPj 


and  Bg 


10 


10 


iPi  (I  -Pi)[.)P2  0 -Pi) 


Bgi  — ( g j/'i  * (I  — P\)^ Pi^ 


586  PaM  -Pi) 


Bgn  - ( g 


/’.’(I  -P^)Pl 


Bsg=P,'°Pi\ 

88 

Now  X Bic  can  be  written 

k «.  84 


88  ^ /I0\  /7\ 

2:  B„=  2;  2:  . bl'll  -P.)'®  ' . + 585-  (in 

84  i 9 i*6  \ ' / \J/ 


The  point  to  be  noted  here  is  that  in  general  S 6*  cniwot  be  written  as  a neat  sum  of  the  form 

k -i 


n.  n. 


V 


z 


-Pi)"'  *'  ■■■  '"'(*  -P")"" 


I,  - «,J  I2-  im  -X 

but  in  general  is  a rather  messy  collection  of  binomial  product  terms  (as  is  shown  by  the  fact 
that  Bgg  cannot  be  included  in  the  summation  on  the  righthand  side  of  equation  (11)).  This 
makes  the  minimization  procedure  needed  to  find 


inf  {p  I S fl*  = 1 — a} 
k i 
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a complicated  computational  exercise,  and  in  fact  effectively  rules  out  its  use  in  practice  for 
cases  when  m is  greater  than  3. 

Turning  now  to  the  problem  of  ordering  the  vectors  x,  three  methods  have  been  suggested. 
Buehler®  noted  that  if  (Xi)  is  an  exact  ai  binomial  lower  confidence  limit  for  pi  then 


n LMi) 

i “ 1 


(12) 


m 

is  a lower  limit  for  p with  confidence  at  least  fl  aj.  This  limit  in  itself  is  not  useful  because 

i - 1 


it  is  generally  considerably  smaller  than  it  needs  to  be  to  achieve  the  confidence 
However,  if  one  takes 


n a, 

i - I 


i = 1,2 m 


one  can  order  the  x’s  in  such  a way  that  the  limits  given  by  (12)  form  an  increasing  set.  This 
order,  suggested  by  Buehler^,  was  presumably  the  one  used  by  Lipow  and  Riley  ‘ ^ in  constructing 
their  tables  using  the  exact  method  given  by  equation  (10).  The  tables  of  Lipow  and  Riley  provide 
lower  limits  on  the  reliability  of  systems  composed  of  one,  two  or  three  components  when  the 
same  number  of  tests  has  been  carried  out  on  each  component. 

The  second  method  for  ordering  is  based  on  the  uniformly  minimum  variance  estimate 
of p and  will  be  referred  to  as  the  “UMVE”  method  (see  Winferbottom  ‘ ').  The  UMVE  estimate 
of  p is 


)5  = n 

i I 

The  vectors  x can  be  ordered  so  that,  if  j < k 

m m 

11  x,)lnt^  II  xtkI'U- 

i I i I 

This  order  has  intuitive  appeal  in  that  if  a set  of  data  x/  give  rise  to  a larger  estimate  of  p than 
another  set  x*  one  would  also  expect  it  to  give  a larger  lower  confidence  limit. 

The  third  method  for  ordering  is  given  by  Winterbottom'  ‘ and  is  called  the  largest  possible 
“LP”  method.  Here 

x.v  ^ (n,,  nj tim) 

and,  given  x,v,  x,v. , is  the  vector  x that  gives  the  largest  Z,j(Xv-i),  and  given  x.v  and  x.v_,,  \s--i 
is  the  vector  giving  the  largest  L^(x,v-2).  and  so  on.  This  method  appears  to  give  a set  of  limits 
that  are  uniformly  large,  but  it  involves  even  more  computation  than  the  other  methods. 

4.  APPROXIMATE  CONFIDENCE  LIMITS  FOR  SYSTEM  RELIABILITY 

It  can  be  seen  from  the  comments  made  above  that  exact  methods  are  usually  too  complex 
to  be  used  in  practice,  and  so  approximate  methods  must  be  resorted  to.  Some  of  the  more 
promising  approximate  methods  that  have  been  proposed  are  discussed  in  this  section. 

The  first  approximate  method  is  attributed  to  Lindstrom  and  Madden  and  is  given  in  Lloyd 
and  Lipow'  p.  226.  It  will  be  referred  to  as  the  LM  method.  The  method  is  very  simple  and 

runs  as  follows.  Given  the  results  .v,-  successes  from  trials  on  component  /,  i = 1,2 m 

one  first  estimates  p by 


p~  II  .x,/n, . 

i " I 

The  collection  of  trials  (/  = 1,2,...,  m)  on  the  components  should  at  least  be  equivalent 
to  n*  = min  (n,)  trials  on  the  whole  system.  If  one  had  carried  out  n*  trials  on  the  system 
and  had  obtained  .t*  — n*  p successes  one  would  have  got  the  same  estimate  of  p as  given  above. 
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Thus  the  data  x*  successes  in  n*  trials  is  taken  as  equivalent  to  the  component  data,  and  the  LM 
lower  confidence  limit  is  the  exact  binomial  limit  based  on  x*  and  n*.  If  x*  is  not  an  integer 
the  limit  is  found  by  interpolation. 

The  LM  method  is  attractive  because  it  is  simple  enough  to  be  executed  by  hand  with  the 
aid  of  a set  of  tables  of  binomial  confidence  limits.  However,  it  does  appear  to  make  rather 
inefficient  use  of  the  data  in  some  cases.  For  instance,  when  n^  — — 20,  jf,  = 15,  = 16 

we  have 

p = 0-6 
n*  = 20 
.V*  = 12 


and  the  LM  lower  0-9  confidence  limit  is  0-433  which  seems  reasonable;  however  when  n,=  40, 
= 20,  .X,  = 30,  X2  = 16  exactly  the  same  results  are  obtained,  which  seems  unreasonable. 
The  second  approximate  method,  due  to  Easterling^,  is  similar  in  concept  in  that  it  deter- 
mines system  results  that  are  deemed  to  be  equivalent  to  the  component  results.  It  has  the 
advantage  of  apparently  making  better  use  of  the  data  at  the  expense  of  being  slightly  more 
complicated.  Easterling  begins  with  the  estimate  of  p given  above  and  notes  that,  by  the  theory 
of  maximum  likelihood  esfimation,  it  has  asymptotic  variance 

m m 

i;  n Pi^VaT(p))  (13) 

j - 1 i = 1 
i * i 

where  p,-  = xjnj 

and  Var  (pj)  = p,- 9, i = 1,  2, . . .,  m- 


An  estimate  of  can  be  constructed  by  replacing  p,  by  p;  in  (13). 

Easterling  then  defines  the  equivalent  set  of  system  results  as  those  values  & and  ft  (repre- 
senting k successes  out  of  it  trials  on  the  system)  that  give  rise  to  the  same  reliability  estimate  f> 
and  the  same  variance  estimate  as  the  component  data.  System  results  ft  give  a reliability 
estimate  .x/ti  and  a variance  estimate  (x//?)(l-.v//5)/«.  Thus  .x  and  fi  must  obey; 

m 

xjfi  = p = n .x,./«,.  (14) 

i = 1 


and  (;?/«)( I — .x//i)/n  = 5^  = fl  Pi^  PiO  — PiM'ti  ■ 

j I i = t 
i ?*) 


(15) 


These  equations  are  easily  solved.  From  (15) 


and  from  (14) 


n = p{l  — p)/a^ 


.?  = /?  p . 


(16) 

(17 


The  lower  confidence  limit  for  system  reliability  is  then  taken  as  the  exact  binomial  limit 
obtained  from  $ and  n.  If  Jc  and  it  are  not  integers  the  limit  can  be  found  either  by  interpolation 
or  by  using  values  of  5 and  n rounded  up  to  the  next  integer.  Following  the  notation  used  by 
both  Easterling^  and  Winterbottom' ‘ the  limits  obtained  by  interpolation  will  be  referred  to 
as  “MML”  limits  and  the  limits  obtained  by  rounding  up  as  “MMLl”  limits. 


As  an  example,  with  the  data  /i,  = 10,  .x,  = 8,  Uj  = 7,  .Xj  = 6,  we  have: 


p = 0-686,  p^  = 0-8,  p2  = 0-8571 

= Pi^  P2O  - PiV’h  : p2^  P\  ('  - Pt'l^'h 


= 0-02295 
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f 

!, 

r From  (16)  « = 9-39 


k 


! 

r 

I 


and  from  (17)  .?  = 6-44. 

Then  the  MMLI  lower  0-9  limit,  based  on  the  rounded  up  values  of  7 successes  from  10  trials, 
is  0-448. 

It  should  be  noted  that  Easterling^  gave  his  theory  in  terms  of  a general  system  with  known 
reliability  function 

P = h(p^,P2,  ■ ■ ■,Pm), 


and  stated  that  the  maximum  likelihood  estimate  /i(^  has  asymptotic  variance 


a 


2 


m 

n 

I 


Var  (pi). 


(18) 


In  the  general  case  the  method  is  approximate  in  the  sense  that  the  asymptotic  variance  is  sub- 
stituted for  the  exact  variance.  In  the  case  under  consideration,  i.e.  the  case  of  a series  system 
where 

rr 

h{pi,p2,  . . ■,Pm)  = 11  Pi, 

i I 

equation  (18)  reduces  to  equation  (13). 

There  are  a number  of  other  methods  based  on  asymptotic  approximations  to  the  distri- 
butions of  certain  statistics.  For  instance,  it  can  be  shown  that  the  maximum  likelihood  estimate 


m 

/)  = II 

i » t 


is  asymptotically  normally  distributed  with  mean  p and  variance  given  by  (13).  This  enables 
one  to  construct  confidence  limits  in  the  usual  way  using  the  norma'  .listribution.  This  particular 
method  suffers  from  the  fact  that  the  normal  distribution  is  symmetrical  and  unbounded  and 
for  small  sample  sizes  is  not  a good  approximation  to  the  true  distributii.,n  of  p which  is  asym- 
metrical (in  general)  and  bounded  on  (0,  I).  The  method  sometimes  gives  confidence  limits  for 
p which  lie  outside  the  interval  (0,  1).  This  method  is  known  as  the  asymptotic  maximum  likeli- 
hood method  (AML). 

Other  methods  based  on  approximating  distributions  are  given  by  Madansky”  and  Mann 
et  al.'*  While  these  appear  to  be  superior  to  the  normal  approximation  method  mentioned 
above  the  comparisons  of  Easterling^.  Winterbottom ' and  Mann  ct  al.'*  indicate  that  the 
methods  of  Lindstrom  and  Madden,  and  Easterling  have  the  most  to  offer  in  cases  where  the 
component  sample  sizes  are  reasonably  small.  Connor  and  Wells ‘ proposed  a method  based 
on  similar  principles  to  the  LM  method  but  requiring  much  more  computation.  Winterbottom' ' 
indicates  that  it  has  no  significant  advantages  over  the  LM  method.  The  reader  is  referred  to 
Mann  el  al.'*  for  descriptions  of  these  methods. 


5.  A NEW  METHOD 

In  this  section  we  propose  a new  approximate  method  for  finding  confidence  limits  for 
the  reliability  of  series  systems. 

The  method  is  similar  to  the  LM  and  MML  MMLI  methods  in  that,  for  the  purpose  of 
constructing  confidence  limits,  the  set  of  component  results  is  replaced  by  “equivalent  system 
results.  Thus,  by  a series  of  operations,  the  data  .v, : ^i,  .v^:  /ij, . . •,  Xm'.  »m  afc  progressively 
reduced  to  .x  : n,  which  is  regarded  as  represenf-.ng  .v  successes  from  ii  trials  on  the  system. 
The  “exact"  binomial  theory  of  section  2 is  then  used  to  give  the  required  confidence  limit. 

Two  concepts  need  to  be  introduced  in  order  to  describe  the  operations  involved  in  re- 
ducing" the  component  results  to  the  equivalent  system  results.  The  first  is  the  idea  of  a sub- 
stitute” result.  Given  .x,-  successes  from  tij  trials  on  component  i,  represented  by  .x,-:  n,-,  a sub- 
stitute" result  is  defined  as  a pair  a : h,  where  a and  h are  any  real  numbers  obeying 
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b < tii . 


For  instance,  4 : 5 is  a substitute  result  for  8 ; 10.  The  second  idea  is  the  “combination”  of 
two  results  xt:  n,  and  xj:  to  one  “equivalent”  set  c : d.  The  results  can  be  combined  if 


(i)  Xj  = nj,  in  which  case  c = Xj  and  d = n^. 


or  (ii)  X]  = in  which  case  c = jr,-  and  d = nj . 


For  instance,  8 : 10  and  6 : 8 can  be  combined  to  give  6 ; 10;  or  8 : 10  and  10  : 12  can  be 
combined  to  give  8:12. 

Consider  now  the  use  of  substitution  and  combination  on  a two  component  system.  Given 
the  results  and  Xz'.nz  two  courses  are  possible; 


(A)  comparing  x,  and  three  cases  arise; 

(i)  Jc,  > rii.  Substitute  riz'  >hnzlx^  for  jr, ; n,  . 

Then  the  results  are  riz'-  n,n2/-^i  and  X2:  nj . 

Combine  these  to  give  .v^;  n^nzlx^  . Thus  the  equivalent  system  results  are  x : n where 


.V  = Xz 


and  n = n^nz|x^ 


(ii)  .V,  < tiz-  Substitute  XiXzInz'.  x^  for  Xz'.  riz 

and  combine  .v, : n,  and  x^XzInz'.  Xi  to  XjXzIfiz^  '>1 


X = XiXz/ftz 


(iii)  .V|  = Hz-  No  substitution  is  necessary  and  the  results  can  immediately  be  combined 
to  .V2 ; rt, 


.r  = .Y2 


and  «=/;,. 


(B)  comparing  .^2  and  u,  (omitting  the  details)  we  have; 

(i)  if  X2  > «!  X = .V,  and  n = n^nzjxz  . 

(ii)  if  .V2  < «!  X = x^xz/n^  and  n = riz , 

(iii)  if  X2  = «,  .Y  = -Y,  and  « = «2  • 


Out  of  the  two  courses  (A)  and  (B)  we  define  the  optimum  one  to  the  be  one  that  produces 
the  greatest  n.  Assuming  without  loss  of  generality  that  n,  ^ /I2,  it  is  easy  to  see  that  course  A 
is  optimum.  Since  n,  > itz  > Xz  course  (B)  gives  /;  = iiz  which  is  less  than  the  n that  (A)  give.s, 
irrespective  of  the  size  of  .y,  . 

Summarizing,  for  a two  component  system  the  results  are  first  ordered  so  that  /i,  ^ tiz^ 
then  a substitution  is  made  so  that  .Y|  = riz  or  X2  = then  the  results  are  combined,  giving 

(i)  if -Y,  > Hz  .Y  = X2  and  n = n,/)2/x,  . 

(ii)  if  X,  = iiz  X = .Y2  and  n = n,  , 

(iii)  if  X,  < Hz  X = XfXzjnz  and  n = ny  . 


According  to  the  rules  of  substitution  and  combination  this  method  produces  equivalent 
system  results  with  the  largest  possible  sample  size. 

As  an  example  consider  the  data  4 : 5 and  10  ; 14.  First  we  reorder  to  make  «,  ^ ttz 
giving  10  : 14  and  4 ; 5.  Comparing  .y,  and  nz  we  find  .y,  is  greater  than  «2  <ifd  so  we  must 
substitute  to  reduce  x,  to  nz-  Thus  5 : 14  x 5/10,  i.e.  5 : 7,  is  substituted  for  10  : 14.  We  now 
have  5 : 7 and  4 : 5.  These  can  be  combined  to  give  4 : 7.  Thus  x = 4 and  n = 7,  and  the 


9 


approximate  lower  0-9  confidence  limit  for  p obtained  from  tables  of  exact  binomial  limits 
given  in  Crow  et  al.'°  is  0-279. 

As  a second  example,  with  the  data  30  : 41,  30  : 36  substitution  for  the  second  component 
gives  30  : 41,  25  ; 30  and  combination  gives  25  : 41  so  that  the  lower  0-9  approximate  limit 
is  0-417. 

The  rationale  behind  the  substitution  procedure  is  that  by  substituting  one  is  throwing 
away  information,  and  therefore  inferences  made  from  substituted  data  should  be  on  the  con- 
servative side  of  the  exact  inferences.  For  instance,  using  the  exact  binomial  theory  given  in 
section  2.  it  can  be  shown  that,  with  only  one  component,  lower  confidence  limits  from  sub- 
stituted data  are  always  smaller  than  those  derived  from  the  original  data. 

The  rationale  behind  the  combination  procedure  is  more  dubious.  It  was  suggested  by  the 
work  of  Winterbottom  and  Verral  which  is  discussed  in  section  7.  The  idea  is  that  the  com- 
ponent results  x,  :n,,  Xj.  where  .x,  = are  exactly  consistent  with  system  results 
where,  unknown  to  the  system  tester,  n,  — x,  failures  were  caused  by  the  first  component,  and 
Xi  — -Xj  were  caused  by  the  second. 

Intuitively  one  feels  that  this  approximate  method  should  give  conservative  limits.  In  all 
cases  investigated  so  far  this  has  proved  to  be  the  case. 

The  procedure  is  easily  generalized  to  the  case  of  m components.  To  begin  with  the  order 
of  reduction  is  specified.  Let  us  assume  the  oroer  is  1,2,  First  x,  :/j,  and  X2:  ni  are 

reduced  to  give  X2':n2'  then  X2‘:n2'  ^nd  x^rnj  are  reduced  to  give  Xj’rnj',  and  so  on. 
For  instance,  the  results  4 ; 5,  9 : 10,  5 ; 6,  7 : 8 give,  successively, 

4 .-  5.56,  5 : 6,  7 .-  8 

3.6  : 6,  7 : 8 

3.6  ; 6.86. 

That  is,  X = 3.6  and  n = 6.86. 

Different  orders  for  reduction  can  give  different  final  values  of  .x  and  n.  An  optimal  order 
is  one  that  gives  the  largest  possible  value  of  n.  It  is  proved  in  Appendix  I that  n,  ^ n2  > ' ‘ ^ 
is  an  optimal  order  (this  has  already  been  shown  to  be  the  case  for  m = 2).  For  instance,  with 
the  data  given  above,  starting  with  the  order 

9 : 10,  7 ; 8,  5 : 6,  4 ; 5 

one  gets 

7 : 8.89,  5 : 6,  4 : 5 

5 : 7.619,  4 : 5 

4 : 7.619. 

Thus  the  proposed  method  is  to  first  reorder  the  component  data  so  that ;/,  > ;;2  > - • - > Wm, 
and  then  to  reduce  it  using  the  rules  of  substitution  and  combination  given  above. 

This  method  is  the  same  as  Lindstrom  and  Madden’s  method  when  the  sample  sizes  are 
equal  (n,  = /)2  = - - - = /im)-  When  the  sample  sizes  are  not  all  equal  it  gives  a greater  value 
of  n in  all  cases  except  when  x,-  = 1 = I,  2, . . .,  m — I,  where  Hm'.  and 

therefore  it  gives  larger  lower  confidence  limits.  As  with  Lindstrom  and  Madden’-.,  method  it 
is  simple  enough  to  be  executed  by  hand  with  the  aid  of  tables  of  exact  binomial  limits. 

6.  A COMPARISON  OF  COMPETING  METHODS 

Features  considered  desirable  for  approximate  methods  are: 

(1)  they  should  give  limits  that  are  reasonably  close  to  the  exact  limits, 

(2)  the  achieved  confidence  levels  should  be  greater  than  the  intended  levels, 

(3)  consistent  with  (2)  above,  the  achieved  confidence  should  be  as  small  as  possible, 

(4)  the  limits  should  be  easy  to  calculate. 

Winterbottom ' ' gives  some  comparisons  of  limits  obtained  by  the  following  approximate 
methods : 


MMLi 

LM 

AML 

Mann 

Madansky 

CW 


a version  of  the  method  of  Easterling  outlined  in  section  4 

the  method  of  Lindstrom  and  Madden  outlined  in  section  4 

the  asymptotic  maximum  likelihood  method  outlined  in  section  4 

the  method  given  by  Mann  ei  a!.'*  and  summarised  in  Winterbottom' 

the  method  of  Madansky ’’ 

the  method  of  Connor  and  Wells'*. 


In  addition  to  these  we  shall  consider; 

MML  another  version  of  the  method  of  Easterling 
SR  the  method  of  “successive  reduction”  proposed  in  Section  5. 

Exact  methods  that  will  be  referred  to  are; 

UMVE  the  exact  method  based  on  ordering  by  the  uniformly  minimum  variance  unbiased 
estimate  of  p. 

LP  the  exact  method  based  on  the  ordering  that  gives  the  “largest  possible”  limits 

(see  Section  3). 

Lipow  & Riley  the  exact  limits  tabulated  in  Lipow  and  Riley 


Only  systems  with  2 components  (m  = 2)  will  be  considered  in  this  section.  This  is  primarily 
because  this  is  by  far  the  simplest  case  computationally;  the  effort  required  rises  enormously 
as  m increases.  We  believe  that  many  features  can  be  successfully  illustrated  by  this  case,  however 
more  work  needs  to  be  done  for  higher  values  of  m to  validate  some  of  the  conclusions. 

To  begin  the  comparison  we  consider  the  question  of  which  of  these  methods  have  feature 
(2)  above,  i.e.  which  methods  actually  achieve  the  confidence  that  they  set  out  to  achieve.  Let 
us  look  first  at  the  confidence  levels  achieved  by  one  of  the  exact  methods,  namely  the  method 
used  by  Lipow  and  Riley‘S  in  constructing  their  tables.  An  extract  from  the  Lipow  and  Riley 
tables  is  shown  in  Table  2.  For  the  case  n,  = Wj  = 10,  Table  2 gives  the  lower  0-9  confidence 
limit  Z,o-9  ^2)  for  as  a function  of  .v,  and  Xj.  For  fixed  values  of  p^  and  the  confidence 

level  actually  achieved  is  the  sum  of  the  probabilities  of  all  pairs  (.v,,  V2)  such  that 


where 


^0-9  -'^2)  ^ P1P2 


Pr{xu  X2)  "'i  (1  (1  —Pi)”^ 


The  confidence  level  achieved  depends  on  the  values  of  p,  and  pj  and  thus  varies  over  the  unit 
square  Thus  it  requires  a 3 dimensional  graph  to  display  the  full  range  of  values.  To  overcome 
this  problem  the  achieved  confidence  levels  have  been  calculated  for  values  of  p,  and  p^  that 
lie  on  the  lines  ' jwn  in  Figure  3.  The  equation  for  this  family  of  lines  is 


Pi  = I ~ “ VpiP2  ^ 


where  / (=  1 or  2)  and  A"  (0  ^ A’  ^ 1)  are  parameters  identifying  particular  lines  in  the  family- 
Figure  4 shows  the  confidence  levels  achieved  by  the  limits  given  in  Table  2 along  the  lines 
determined  by  / = 1 and  K — 1,  0-8,  0- 1 and  0-01.  In  these  figures  the  abscissa  p equals  P\Pi. 

Figure  4 shows  “cuts”  through  the  surface  of  achieved  confidence  over  the  unit  square. 
It  can  be  seen  that  the  surface  comes  down  and  touches  the  plane  of  confidence  0-9  at  a series 
of  points  over  the  unit  square,  just  as  in  the  I dimensional  case  shown  in  Figure  1 the  curve 
comes  down  and  touches  the  line  of  confidence  0-9.  There  is  at  least  one  such  touch  point  in 
the  plane  for  each  confidence  limit  shown  in  Table  2,  and  this  point  lies  somewhere  on  the  line 

P1P2  = that  limit. 

For  instance  the  value  0-5504  is  a limit  shown  in  Table  2 (it  corresponds  to  .v,  = 8,  = 10 

and  .V,  = 10,  X2  = 8).  The  line 

PxPi  = 0-5504 
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is  shown  in  Figure  5 and  the  surface  of  achieved  confidence  touches  the  plane  of  confidence 
0-9  at  the  points  = 0-5504,  pi  = 1 and  /),  = 1,  = 0-5504.  Similarly  the  value  0-4133 

is  another  limit  shown  in  Table  2,  and  surface  touches  the  plane  at  the  point  P\  — Pz  = 0-6429 
which  lies  on  the  line 

PiPi  = 0-4133. 

Some  other  touch  points  are  also  shown  in  Figure  5.  Although  this  case  gives  the  impression 
that  all  the  touch  points  lie  on  one  of  the  lines  /»i  = I,  />2  = 1 or  p^  = P2,  this  is  not  true;  in 
general  they  lie  anywhere  in  the  unit  square. 

Returning  to  the  consideration  of  approximate  methods,  if  the  surface  of  achieved  confidence 
for  a particular  method  goes  down  below  the  0-9  plane  at  any  point  then  it  does  not  achieve  the 
desired  confidence,  i.e.  it  does  not  satisfy  criterion  2.  As  an  example  consider  the  MMLI  method 
when  /;,  = 10,  Wj  = 7.  The  cut  given  by  A"  = 1 is  shown  in  Figure  6.  Clearly  the  MMLI  method 
does  not  guarantee  to  achieve  the  desired  confidence. 

To  establish  that  a particular  method  does  not  guarantee  the  desired  confidence,  all  one 
needs  is  one  example  of  it  not  doing  so.  To  establish  that  a method  does  achieve  the  desired 
confidence  one  needs  to  prove  the  truth  of  this  proposition  in  all  possible  cases.  As  far  as  we 
are  aware  no  such  proof  exists  for  any  of  the  approximate  methods.  However,  after  examination 
of  a number  of  cases  it  seems  likely  that  some  methods  do  have  this  property. 

Table  1 of  Winterbottom"  provides  examples  that  show  that  the  methods  MMLI,  AML 
Mann  and  Madansky  do  not  always  achieve  the  desired  level  of  confidence.  This  follows  from 
the  fact  that,  with  the  same  ordering  as  one  of  the  exact  methods  (see  Section  3 for  a discussion 
of  ordering),  there  are  cases  where  the  approximate  limits  are  greater  than  the  exact  limits. 
These  cases  are  given  in  Table  3. 

The  methods  for  which  we  do  not  know  of  any  such  examples  are  MML,  LM,  CW  and  SR. 

The  LM  and  CW  methods  will  not  be  considered  further;  the  LM  method  because,  as  was 
pointed  out  in  Section  5,  it  is  dominated  by  the  SR  method  (dominated  in  the  sense  of  always 
giving  equal  or  smaller  limits  than),  and  the  CW  method  because  the  comparisons  of  Winter- 
bottom*'  show  that  it  is  very  similar  in  performance  to  the  LM  method. 

The  MML  and  SR  methods  appear  to  be  the  two  that  give  the  largest  limits  consistent 
with  feature  (2),  which  means  of  course  that  they  are  best  in  terms  of  feature  (3).  Both  methods 
generate  “equivalent"  system  results,  x successes  out  of  n trials,  where  .v  and  n are  not 
necessarily  integers.  Therefore  to  obtain  confidence  limits  from  ordinary  tables  some 
form  of  interpolation  must  be  used.  The  one  used  to  generate  the  results  discussed  below  is  as 
follows.  Let  [y]  be  the  integer  part  of  y.  Let  LJ^x,  n)  be  the  * lower  confidence  limit  given  by 
equation  (4)  (the  notation  has  been  altered  slightly  from  Lj,x)  to  Z,,(.v,  n)).  Then,  if  .r  and  n 
are  not  integers,  the  limit  is  equal  to 

a ■ (n  - [«J)  (6  — a) 

where  a = (x  - [x])  (L^{[x]  t 1,  [«])  - L^(lx],  [n]))  t ^^.([.x],  [n]) 

and  b ~{x  — [x])  (M[x]  ^ I,  [«]  + I)  — La([x],  [n]  -r  I))  - M[x],  [n]  - 1). 

An  alternative  form  of  interpolation  would  have  been  to  use  (5),  since  /(.v,  a,^)  is  defined  for 
real  values  of  .v,  a and 

To  enable  comparison  with  results  given  by  Winterbottom*  * some  of  the  limits  produced 
by  the  MML  and  SR  methods  in  the  cases  n,  = 10,  /ij  = 7 and  n,  = 10,  rij  = 10  are  given  in 
Table  4.  The  limits  given  by  the  exact  LP  and  UMVE  methods  are  also  given. 

Easterling^  pointed  out  that  the  MML  method  runs  into  difficulties  in  cases  where  no 
failures  are  observed.  In  his  analysis  he  used  the  rule  “For  the  case  of  no  failures  the  pseudo- 
sample size  was  taken  to  be  min  («,,  /ij)'  ’ This  statement  is  slightly  ambiguous,  it  could  mean: 

(a)  if  X,  = n,  and  Xi  = 'h,  let  n = min  (n,,  112), 
or 

(b)  if  .X,  = n,  or  .X2  = /I2,  let  n = min  (n,,  n2)- 

Interpretation  (a)  would  certainly  lead  to  a rule  where  the  achieved  confidence  was  sometimes 
less  than  the  intended  confidence.  Taking  an  extreme  example,  with  n,  = 30.  1/2  = ’ l^*^  results 
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x,  = 29,  ,V2  = 2 would  give  .{  = 29,  n = 30  and  a lower  0-9  confidence  limit  of  0-876.  How- 
ever the  exact  limit  must  be  less  than  or  equal  to  that  obtained  from  2 successes  in  2 trials, 
viz.  0-316. 

For  this  reason  we  have  used  interpretation  (b).  I hus  the  results  n,  — 30,  rij  = 2,  x,  = 29 
X2  = 2 give  n — 2,  p = 0-967,  x = 1 -933  and  lower  0-9  limit  of  0-298. 

An  investigation  of  the  performance  of  the  SR  method  was  carried  out.  The  cases  investi- 
gated are  listed  in  Table  5.  In  no  case  was  the  achieved  confidence  level  lower  than  the  intended 
level.  As  a typical  example  of  the  results  achieved.  Figure  7 shows  four  “cuts”  through  the 
confidence  surface  for  the  case  n,  = 20,  Uj  = '0.  Figure  8 shows  a comparison  of  three  “cuts” 
for  the  SR  and  MML  methods  when  /i,  = 10,  112  = 7.  For  the  case  A = 1 the  SR  method  is 
better  tor  values  of  p above  0-5,  the  two  methods  are  roughly  equal  for  0-17  $ p $ 0-5,  and 
the  MML  method  is  better  for  p less  than  0-17.  Since  in  practice  one  is  rarely  interested  in 
systems  with  p < 0-5  the  SR  method  seems  to  have  the  edge  on  this  comparison.  In  the  case  1 = 1, 
K = 0 ii\  the  performances  are  virtually  identical.  When  / = 2,  K = 0-01  (in  this  case  />2  1. 

and  p varies  with  p^  from  0 to  1)  the  MML  method  performs  poorly.  This  case  highlights  the 
difficulty  encountered  by  the  MML  method  when  there  are  no  failures  on  the  component  with 
the  smallest  sample  size. 

Figure  9 shows  a further  comparison  of  the  two  methods  when  /i,  = /i2  = 10.  The  MML 
method  is  superior  in  this  case. 

Based  on  a fairly  small  amount  of  evidence  our  tentative  conclusions  are  that  the  MML 
method  performs  better  when  the  sample  sizes  are  equal  or  nearly  so,  and  the  SR  method 
performs  better  when  the  sample  sizes  differ  significantly.  However  more  evidence  is  needed, 
especially  on  structures  with  more  than  two  components. 

7.  A SEQUENTIAL  PROCEDURE  FOR  SERIES  AND  PARALLEL  SYSTEMS 

The  discussion  in  the  previous  sections  has  been  concerned  with  the  problem  of  constructing 
confidence  limits  for  system  reliability  with  data  obtained  from  fixed  sample  size  tests  of  the 
components.  Clearly  this  is  a difficult  problem.  The  problem  can  be  avoided  if  the  sample  sizes 
arc  not  fixed  in  advance,  but  are  chosen  sequentially  throughout  the  testing  of  the  components. 
Winterbottom  and  Verrall*  give  such  a scheme. 

The  scheme  for  testing  the  components  of  a series  system  is  as  follows.  Test  component  1 
n times  giving  x,  successes  and  n,  — x,  failures.  Test  component  2 iij  = x,  times  giving  x> 
successes  and  1I2  — X2  failures,  and  so  on,  with  component  1 being  tested  11, ■ = .Xj  , times  giving 

.Xj  successes  and  iij  — x.  failures  (1  = 2,  3 m).  Winterbottom  and  Verrall*  show  that  Xm  is 

an  observation  on  a binomial  random  variable  with  probability  p and  sample  size  n.  Confidence 
limits  for  p can  be  constructed  from  x,„  and  n by  the  method  given  in  Section  2.  Thus  the  rule 
used  for  testing  components  linked  in  series  is  to  use  the  number  of  successes  obtained  for 
component  / as  the  sample  size  for  component  ; • 1. 

Winterbottom  and  Verrall'*  give  a similar  rule  for  testing  a system  whose  components  are 
linked  in  parallel,  i.e.  a system  with  reliability 

m 

P = \ — 11  (I  — p). 

i I 

The  rule  is  that  the  sample  size  for  component  1 is  the  number  of  failures  obtained  on  com- 
ponent 1 — 1. 

These  two  rules  can  be  combined  for  any  system  that  can  be  broken  down  into  blocks 
consisting  of  units  arranged  in  series  or  parallel.  The  results  can  still  be  expressed  as  an  obser- 
vation on  a binomial  random  variable  with  sample  size  n and  probability  p. 

Often  it  is  impossible  for  practical  reasons  to  be  able  to  run  sequential  tests  of  this  type. 
In  those  cases  fixed  sample  size  tests  must  be  used  and  the  methods  discussed  in  the  previous 
sections  apply.  However,  in  cases  where  sequential  tests  can  be  used,  the  scheme  of  Winterbottom 
and  Verrall  is  efficient  in  the  sen.se  that  it  minimizes  the  number  of  tests  required  and  it  provides 
data  from  which  confidence  intervals  can  be  calculated  easily  by  exact  methods. 
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8.  A GENERAL  SEQUENTIAL  PROCEDURE 

Winterbottom  and  Verrall’s  scheme  was  generalized  by  Preston’  to  deal  with  any  coherent 
system  whose  structure  function  is  known.  This  procedure  determines  the  number  («,)  of  tests 
that  must  be  performed  on  each  component,  and  schedules  these  tests  in  such  a way  that  the 
results  can  be  expressed  as  an  observation  on  a binomial  random  variable  with  sample  size  n 
and  probability  p. 

We  set  out  to  perform  the  equivalent  of  n tests  on  the  whole  system.  It  will  be  convenient 
to  refer  to  these  as  single  tests  on  n identical  ‘systems’.  Thus  if  jr,;(=  1 or  0)  is  the  result  of 
the  jth  test  on  the  ith  component  the  vector  (.v,/.  .Vj/, . . .,  Xij, . . . Xm))  will  be  the  result  of  the 
tests  on  ‘system’  j. 

It  is  not  always  necessary  to  carry  out  n tests  on  all  components,  for  the  following  reason. 
Suppose  we  have  just  finished  testing  component  r and  that  the  results  of  the  tests  on  the  com- 
ponents of  ‘system’  j so  far  obtained  are  (.v,;,  x^j, . . .,  Xrj).  If 

/(.V|J,  Xij Xrj,  Cr,  1,  . . .,  Cm)  = 1 

for  all  values  of  the  vector  (Zr  + |, . . .,  Zm)  where  z,  = 0 or  1,  then  we  know  that  ‘system’  j will 
be  assessed  as  reliable,  irrespective  of  the  results  of  the  tests  on  components  r ^ 

Thus  it  is  not  necessary  to  test  these  components  for  ‘system’  / Similarly,  if 

f (-'  ij'  • ■ •'  ^rj,  Zr  . Zm)  = 0 

for  all  (zr,  (,...,  Zm)  ‘system’/  has  failed  the  system  test  and  its  remaining  components  do  not 
need  to  be  tested.  Thus,  at  stage  / of  the  sequential  testing  procedure  we  define  three  classes  of 
‘systems’,  those  that  have  passed  the  test,  those  that  have  failed,  and  those  whose  outcome  is 
not  yet  determined.  We  denote  these  as  follows: 

‘systems’  /,  7 e -4  j,  have  passed 
‘systems’ 7, 7 6 fi,-,  have  failed 
‘systems’ 7. 7 e C,-,  need  further  testing. 

The  sequential  procedure  runs  as  follows.  Test  component  I n times  with  results 
.x,,,  x,2 -Vin-  Let  /I,  be  the  set  of  integers  7 such  that 

/(•X|;,  Zj,  Zj Zm)  = 1 

for  all  (zj Zm);  and  let  B,  be  the  set  of  integers 7 such  that 

f (Xfj,  Z2 Zm)  = 0 


for  all  (Z2 Zm)  {A,  and  B,  will  often  be  null  sets).  Let  C,  be  the  set  of  integers  in  the  range 

1,2,...,  ti,  not  contained  in  /(,  or  S,.  Let  .s,  be  the  number  of  elements  in  C,. 

Test  component  2 .v,  times  and  record  the  results  as  -Vjy,  7 e C,.  Let  £>2  be  the  set  of  integers 
7, 7 6 C,  such  that 

/(•V|j,  .V2J,  Zj,  . . .,  Zm)  ==  1 


for  all  (Zj Zm),  and  let  £2  the  set  of  integers  7,76  C,  such  that 


f{Xjj,  X2j,  Z3,  . . .,  Zm)  — 0 


for  all  (Z3 Zm).  Let 

A2  = A,UD2, 

B2  = B^VE2, 

^2  = C.  - D2  - £2. 

and  let  S2  be  the  number  of  elements  in  £2- 

Continuing  in  this  manner,  the  ilh  step  is  as  follows.  Test  component  /'  .Vj  , times  and  record 
the  results  .x,/, /eC,  ,.  Let  O,-  be  the  set  of  integers  j.  J £ £,■  |,  such  that 

/(.V,> -XU.  z,., Zm)=l  (19) 


I 


I 


J 


1 


i 


i 
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for  all  (z,+i, . ■ Zm),  and  let  £j  be  the  set  of  integers  j,je  C,.,,  such  that 

f^X^, Xil,  Zj+„  . . Zm)  = 0 

for  all  (z.+i, . . Zm).  Let 


Aj  — Aj^iUDi, 

B = Bi_tUE„ 

C = Cj.,  Dj  £(, 


and  let  ij  be  the  number  of  elements  in  C,.  This  procedure  is  followed  for  / = 2,  3 m. 

The  probability  of  a ‘system’  being  classified  in  the  set  Am  on  completion  of  testing  is  just 
the  probability  of  the  system  passing  a system  test,  i.e.  the  probability  p.  Let  x be  the  number 
of  elements  in  y4m  on  completion  of  testing.  Then  x is  an  observation  on  a binomial  random 
variable  with  sample  size  n and  probability  p.  The  proof  of  this  proposition  is  given  in  Appendix  2. 
It  is  easy  to  show  that  this  procedure  reduces  to  the  one  given  by  Winterbottom  and  Verrall 
for  a series  or  parallel  system. 


Example:  The  structure  function  for  a bridge  network  (see  Figure  10)  is 
/O’l.Ta.Ta.n.  J’s)  = 1 - (I  -TiTzX'  -3'3>'4)(>  - J’JVsKl 


This  function  is  represented  in  tabular  form  in  Table  6.  The  brackets  indicate  that  the  values 
of  the  component  indicators  to  the  left  determine  the  value  of  y irrespective  of  the  values  of 
the  component  indicators  to  the  right.  For  instance,  from  rows  21  to  24,  it  can  be  seen  that 
if  y,  = 0,  ^2  = L and  yj  = 0, 


'4.  Z5)  = 0 

for  all  (Z4,  Z5). 

Suppose  n = 10  and  the  results  for  component  I are  1,  I,  1,  0,  1,  1,  0,  1,  1,  1,  i.e. 

X,,  = 1,  x,2  = 1,  x,3  = 1,  x,4  = 0 etc. 


The  sets  A,  and  B,  are  null. 


C,  = {I,  2,  3,  4,  5,  6,  7,  8,  9,  10} 
and  = 10. 


Component  2 is  tested  10  times  giving  1,  1,0,0,  I,  I,  1,0,  I,  1. 

Pairing-off  with  the  results  for  component  1 we  have 

j 123456789  10 
I I I 0 1 I 0 I I I 

X2J  I 10  0 1 1 10  1 1 

Then  ^2=  {1,  2,  5,  6,  9,  10} 

A2  = D2 
£2  is  null 
£2  is  null 
£2=  {3,  4,  7,  8} 
and  S2=  4. 

Component  3 is  tested  4 times  giving  0,0,  1,  1.  The  results  for  the  systems  in  C2  are 

j 3 4 7 8 

Xt,\  0 0 1 

.V2i0  0 I 0 
x,>0  0 1 1 
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I 
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Thus  D3  is  null 

/4j  = A2 

Ei  = {4} 

B,  = {4} 

C,  = {3,  7,  8} 

and  Sj  = 3. 

Component  4 is  tested  three  times  giving  1,1,0.  The  results  for  the  systems  in  Cj  are 

j 3 7 8 
1 0 1 
0 1 0 
XjJO  I 1 

X4;l  1 0 

Thus  £>4  = {7} 

^4  ={1.2,  5,  6,  7.  9,  10} 

£■4  is  null 
B4  =53 
C4  = {3,  8} 
ands  54  = 2. 

Component  5 is  tested  twice  giving  1,0.  The  results  for  the  systems  in  C4  are 

J 3 8 

Xli  1 1 

X2J  0 0 

XjjO  I 
X4J  1 0 

AT,;  1 0 

Thus  D,  = {3} 

A,={1,2,  3,  5,  6,  7,  9,  10} 
and  X = 8. 


This  set  of  component  results,  namely 
component 

1 

2 

3 

4 

5 


no.  of  trials 
10 
10 
4 
3 
2 


no.  of  successes 
8 
7 
2 
2 
1 


gives  the  equivalent  system  result  of  8 successes  out  of  10  trials.  A 90%  confidence  interval  for  p, 
calculated  by  the  method  given  in  Section  2,  is  (0-550,  1.). 
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APPENDIX  1 


Assuming  without  loss  of  generality  that  n,  ^ > Wm.  in  this  appendix  it  is  proved 

that,  using  the  methods  of  substitution  and  combination  described  in  Section  5,  1,2, . . w 
is  an  optimal  order.  That  is,  it  gives  the  largest  possible  value  of  n. 

The  methods  of  substitution  and  combination,  when  applied  together  and  in  that  order, 
will  be  referred  to  as  the  method  of  “reduction”.  Thus  we  speak  of  reducing  x,:  n,,  xj:  nj  to 
xt-  «*,  and  n*  is  called  the  “result”  of  the  reduction. 

A general  order  will  be  denoted  by  /(I),  /(2), . . .,  i(m)  where  (i(l),  /(2), . . .,  i(ni))  is  a per- 
mutation of  (1,  2, . . .,  m).  The  set  (/),(,).  • • •,  will  be  referred  to  as  the  sample  size 

vector.  The  reduction  process  is  always  carried  out  from  left  to  right,  so  that  successive  sample 
size  vectors  will  be 

Ni  = n,(2(, . . . n,(OT)) 

N2  = (n*2>  ”i(3)>  • • •• 

N3  = (n'},  ni(4),  . . .,  numt) 

where  N,  has  m components,  N2  has  m — 1 components;  and  so  on,  and  n'2  is  the  result  of 
reducing  .v,(i);  Jr,(2):  n,(2)  etc. 

THEOREM  I When  w = 2 an  optimal  order  is  1,  2 and  the  substitution  rules  are: 

(i)  if  X,  > substitute  n2-  fof  ^i-  ”1 

(ii)  if  X|  = ^2  no  substitution  is  required 

(iii)  if.e,  < substitute  x,X2/n2  ■ for  Xi’.  n2- 
Proof : The  proof  is  given  in  the  text  of  Section  5. 

THEOREM  2 The  data  x, : n,,  X2:  ^2  give  an  equal  or  better  result  than  x, ; X2:  where 


A*i  X 

— = — and  /J2  ^ ”i  ^ "i' 
n,  n, 

Proof : 

(i)  If  .f,  > /J2'  n^,  X2'.n2  give  the  result  n = n,  n2/-*'il  and  x,:n,,  X2- 12  give  the 

result  n = rt,/;2/X|  > n. 

(ii)  If  .V,  = /J2-  X2-n2  give  n = ii,  and  x, X2'- n2  give 

n = nin2/x,  (since  x,  > .?,  = ^2) 

= »lXi/Xi 

= «i 

= n. 

(iii)  If  X,  < /J2.  -'1 : «i.  X2:»2  give  n = n,  and  x, : n,,  -Vj:  /I2  give 

n ? rttn2/-*i 

> fiyxjxi 

= "1 
= n. 

THEOREM  3 When  w = 3 an  optimal  order  is  1,2,  3. 

Proof;  An  order  will  be  said  to  dominate  another  order  if  it  gives  at  least  as  great  and  possibly 
greater  value  of  n.  The  possible  orders  are: 

A 1,2,3 

B 1,3,2 

C 2,  1,  3 

D 2,  3,  1 

E 3,1,2 

F 3,  2,  I 
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Now,  from  Theorem  1 the  order  1,  2 dominates  the  order  2,  1.  Applying  theorem  2 it  follows 
that  1,  2,  3 dominates,  2,  1,  3,  i.e.  C is  dominated  by  A.  Similarly  E is  dominated  by  B and  F 
by  D.  This  leaves 


A 1,2,3 

B 1,3,2 

D 2,  3,  1 

An  exhaustive  analysis  of  the  results  obtained  from  each  of  these  ordeis  is  given  in  Table  7. 
Examination  of  every  case  shows  that  order  A always  gives  a value  of  n as  great  as  or  greater 
than  orders  B and  D. 

THEOREM  4 For  any  m > 2 the  order  1,  2, . . .,  m is  an  optimal  order. 

Proof:  The  proof  is  by  induction.  Theorems  1 and  3 have  established  its  truth  for  w = 2 and 
m = 3.  Assuming  its  truth  for  m — 1 components  it  will  be  shown  to  be  true  for  m components, 
where  m > 4. 

It  is  obvious  that  at  least  one  optimal  order  must  exist.  Let  /(I),  /(2), . . .,  i(m)  be  such  an 
order.  Now  consider  the  first  m - 1 components  /(I),  /(2),  . . .,  i{m  — 1).  These  will  be  success- 
ively reduced  to  give  the  result  n'm-x-  However,  by  the  inductive  assumption,  the  order 
1,  2, . . .,  /(m)  — 1,  /(m)  i- 1, . . .,  m must  give  at  least  as  good  a result.  Further,  by  Theorem  2, 
this  must  combine  with  component  i(m)  to  give  at  least  as  good  a result  for  the  whole  system. 

That  is,  the  order  1,  2, . . ., /(m)-l,  /(w)+l m,  i(m)  must  also  be  optimal.  Now  three 

cases  arise. 

Case  (a)  i{m)  > 2. 

Beginning  with  the  optimal  order  1,2,...,  /(m)  — 1,  i(m)  • 1, . . .,  m.  i{m)  the  first  reduction 
gives  the  sample  size  vector 

in' 2<  • • -I  'hn.  'I, (mi) 

where  n'2  > /I3.  By  the  inductive  assumption  from  here  on  the  order  2,  3, . . .,  m must  be  optimal. 

But  clearly  the  initial  order  1,2 m would  have  given  an  identical  result,  so  it  too  must  be 

optimal. 

Case  (b)  i(m)  = 2. 

Beginning  with  the  optimal  order  1,3,4 m,  2 the  first  reduction  gives  sample  size 

vector  (//'j,  «4 nm,  ih).  Depending  on  which  of  n' 2 and  is  largest,  since  there  are  now 

m — 1 components,  the  order  corresponding  to  either  (/I'j,  "3- • • ■<  "m)  or  ”3- 

nm)  is  optimal.  Continuing  with  the  appropriate  one  of  these,  the  next  reduction  gives 

sample  size  vector  /?4,, . .,  «,n).  Now  compare  this  with  what  would  have  been  obtained 
starting  with  the  initial  order  1,2 m.  Then  n ' 3 was  obtained  by  reducing  the  first  three  com- 

ponents in  the  order  I,  3,  2,  and  by  Theorem  3 this  must  give  a result  less  than  or  equal  to  that 
obtained  from  the  order  1,2,  3.  Repeated  application  of  Theorem  2 then  shows  that  the 
final  result  obtained  from  1,2,  ...,/m  must  be  at  least  as  good  as  that  obtained  from  the 
optimal  order  1,3,4 m,  2.  Thus  1,2 m must  also  be  optimal. 

Case  (c)  i{ni)  = 1. 

Beginning  with  the  optimal  order  2,  3 wi,  1 the  first  reduction  gives  sample  size  vector 

(m'j,  «4,  «5 «m.  'll). 

Since  there  are  now  ni  — 1 components  the  order  corresponding  to 

('ll.  nS.  "4.  "s Urn) 

must  be  optimal.  The  next  reduction  gives 

('l'3,  'I4,  Us "m). 

Here  n'j  was  obtained  from  reducing  the  first  three  components  in  the  order  2,  3,  I . By  Theorem  3 
the  order  I,  2,  3 must  be  at  least  as  good,  and  repealed  application  of  Theorem  2 shows  that 
the  initial  order  I,  2, . . .,  ni  must  also  be  optimal. 

Thus  in  every  case  the  order  1,2,...'  is  an  optimal  order.  This  completes  the  proof. 
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Additional  Notation 


APPENDIX  2 


i,j,k,r,t  integer  indices 


F]  binary  vector;  m elements;  general  element /y,  /=  1,2 m;  the  vectors  /j, 

y = \,  2, . . 2’”  are  arranged  in  lexicographic  decreasing  order  i.e. 


F.  =(1,  1,..., 

1,  1,  1) 

F,  = (1,1,..., 

1,  1,0) 

F3=(1,  I,..., 

1,0,  1) 

F4  = (1,1...., 

1,0,0) 

F'} 

binary  vector;  r elements;  general  element  /,;,  / = 1,2 r;  F’j  is  the  vector  Fj 

truncated  at  the  rth  place:  F'j  — (/,j, . 

. .,fri)  and  in  particular  F";  = F/. 

Hk 

binary  vector;  b(k)  elements;  defined 

as  follows: 

//,  = (I),  H,  = (0),  H,  = (I,  I),  = (I,  0) 

H,  = (0,  1),  //,  = (0,  0),  //,  = (I.  1,  1), . . .; 

i.e.  the  number  of  elements  starts  at  I and  increases  in  steps  of  I,  and  for  a given 
number  of  elements,  all  the  binary  vectors  are  listed  in  lexicographic  decreasing  order. 

b{k)  number  of  elements  in  //*■;  b(k)  is  the  integer  part  of  log^lA-  r 1). 

O'  the  set  of  indices ysuch  that/(/v)  = I (recall  that/(  ) is  the  reliability  function);  i.e. 

/(f})=  \JeG;f(F,)  = 0,j^G. 

Gj,  V„  Pi  the  sets  G„  the  vectors  K,-,  and  the  scalars  p,,  / = 1,2 h,  are  defined  recursively 

as  follows:  let  k be  the  smallest  integer  such  that/(//t,  . . .,  Zm)  = 1 for  all 

('6(*l  tl>  • • ■' ^m)'  (Al) 

Define  C,  as  the  set  of  indices  / such  that 

F?'*’  ^ Hic.  (A2) 

Define  p,  = h(k)  and  F,  = //*.  Find  the  next  smallest  integer  k such  that  (Al) 
holds,  define  G^  as  the  set  of  indices  J such  that  (A2)  holds,  define  pj  = b(,k)  and 
= Hic\  and  so  on.  As  an  example  consider  the  2-out-of-4  voting  system  given  in 
Sec.  8.  From  Table  6 we  see  that  the  smallest  k such  that  (Al)  holds  is  /c  = 3,  and 
//j  =(l,  1). 

Then  G,  = [I.  2,  3,  4,  5,  6,  7,  8],  p,  = 2,  F,  = (I,  1). 

The  next  smallest  A such  that  (Al)  holds  is  A = 12,  Hiz  = (0,  1, 0). 

Then  G,  = [21,  22,  23,  24],  pj  = 3,  = (0,  1,0). 

Prcceeding  in  a similar  fashion  we  find  that 

G4  = [ 9,10],  p4  = 4,  F4  = (1,0,  1,1) 

G,  = [II,  12],  p,  =4,  F,  = (l,0,  1,0) 

G,  = [15,  16],  p,  = 4,  F,  = (1,0,0,0) 

G,  = [17,  18],  p^  = 4,  F,  = (0,  I,  I,  1),  and  so  on. 

h number  of  sets  G,. 
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vti  elements  of  the  vector  K,;  ^ = 1,  2, . . p,;  / = 1,2,..  .,  h. 

Si  set  of  indices  j such  that  pj  = / for  / = 1,  2, . . .,  m.  In  the  example  given  in  the 

definition  of  G,- 

S,  is  null.  Si  = [1],  Sj  = [2,  3],  S4  = [4,  5,  6,  7,  8,  9],  Sj  = [10,  11,12]. 

<IKp,z)  pUl  -P)'~^ 

Proposition 

The  probability  of  a ‘system’  being  classified  in  the  set  /<m  is  the  probaility  of  the  system’s 
being  good,  i.e.  the  probability  R. 

Proof 

The  probability  of  the  component  configuration  represented  by  any  binary  vector  Fj  is 

m 

/’r{T,}=  n ^ 

k = I 

G is  the  set  of  all  component  configurations  that  make  the  system  good.  The  system  reliability, 
/?,  is  the  sum  of  the  probabilities  of  all  such  component  configurations,  i.e. 


«=  2 Pr{Fi\. 

jeO 


It  follows  from  the  definitions  that  the  sets  G,-,  i — I,  2, . . .,  A,  are  a mutually  exclusive  and 
exhaustive  collection  of  subsets  of  G,  i.e. 

h 

G=  u G,. 

i - 1 

Thus 

h 

/?=  2:  2 Pr[Fi}.  (A3) 

i-l  ieCj 


Further,  from  the  definition  of  Gj,  the  vectors  F^,y6  Gj,  have  their  first  pi  elements  in  common; 
in  fact  these  first  p,-  elements  constitute  the  vector  F,-,  i.e., 

ft}  = Vkh  k = 1,2 Pi  for  j e Gj. 

Thus 

m 

2 Pr{F,}=  2 n ^RkJki) 

jcCj  j6G.  k-l 


Pj  m 

= 2 n .>i)  n >HRkJki) 

j € Gj  k I k Pj  • I 

m 

= n URk^vki)  x r <KRk.fki) 

k I j€Gj  k Pj-^l 

Pi 

= 11  'f‘(Ric,yki)  (A4) 

k I 

The  last  step  follows  from  the  fact  that,  by  the  definition  of  Gj,  the  vectors  ■ ■ .,/mj), 

J^G,,  constitute  a complete  enumeration  of  the  binary  vectors  with  m — pj  elements,  which 
implies 

m 

I n iM.Rk.fkt)  — 1- 

icGj  k - Pj  ^ I 
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TABLE  1 

Exact  and  approximate  lower  90%  confidence  limits  for  p when  n = 10. 


-V 

0 

0- 

0- 

1 

00105 

0- 

2 

0 0545 

0-038 

3 

01158 

0-114 

4 

01876 

0-201 

5 

0-2673 

0-297 

6 

0-3542 

0-401 

7 

0-4483 

0-514 

8 

0-5504 

0-638 

9 

0-6632 

0-778 

10 

0-7943 

1-000 
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TABLE  3 


Cases  where  the  approximate  lower  0 - 9 confidence  limits  are  greater  than  the  exact  limits. 


”l 

! 

1 

Ordering* 

1 

Exact 

Limit 

Approximate 

Method 

Approximate 

Limit 

10 

7 

9 

5 

LP 

0-388 

MMLl 

0-401 

10 

7 

10 

5 

LP 

0-404 

AML 

0-495 

10 

10 

9 

9 

UMVE 

0-607 

Mann  ' 

0-615 

10 

7 

10 

5 

LP 

0-404 

Madansky 

0-475 

* This  column  gives  the  ordering  used  in  the  determination  of  the  exact  limits  (see  Section  3 
for  a discussion  of  ordering). 


TABLE  6 

Structure  function  for  a bridge  network. 


row 

>-1 

yz 

yi 

y* 

T's 

y 

1 

I] 

I 

1 

1 

I 

2 

1 

1 

1 

‘ 1 

0 

1 

3 

1 

1 

1 

0 

1 

1 

4 

1 

1 

0 

0 

1 

5 

I 

0 

I 

1 

1 

6 

1 

0 

1 

0 

1 

7 

1 

0 

’ 0 

1 

1 

8 

1 

0 

0 

0 

1 

9 

1 

0 

1 

11 

1 

1 

10 

1 

0 

1 

ij 

0 

1 

II 

1 

0 

1 

01 

1 

0 

12 

1 

0 

1 

of 

0 

0 

13 

1 

0 

0 

1 

1 

1 

14 

1 

0 

0 

1 

0 

0 

15 

1 

0 

0 

0| 

1 

0 

16 

1 

0 

0 

0 

0 

17 

0 

I 

1 

' '1 

I 

1 

18 

0 

1 

1 

IJ 

0 

1 

19 

0 

1 

1 

0 

1 

1 

20 

0 

1 

1 

0 

0 

0 

21 

0 

1 

1 

1 

0 

22 

0 

1 

0 

1 

0 

0 

23 

0 

1 

0 

0 

1 

0 

24 

0 

1 

oj 

0 

0 

0 

25 

0 

0 

1 

ll 

1 

1 

26 

0 

0 

1 

IJ 

0 

I 

27 

0 

0 

1 

0( 

1 

0 

28 

0 

0 

1 

0) 

0 

0 

29 

0 

0 

1 

1 

0 

30 

0 

0 

0 

• 

0 

0 

31 

0 

0 

0 

0 

1 

0 

32 

0 

0 

0 

0 

0 

0 

Condition 


X,  ^ til 
Xl  > «J 
n,nj/.v,  ^ /Jj 

.V,  > ni 
Xi  > rti 
■V2  ^ nirt3/-V, 

.V,  > /Ij 

A-j  > Wj 
"i«3/Ai  < Aj 

X,  ^ ^2 
X2  < "3 
Wi«3/Ai  > »! 

X,  ? /J2 

X2  < n, 

«I«3/Xl  < "2 
X,  < ^2 

X,X2//l2  > /I3 

«1«3/Xl  > «2 

X,  < «2 
X,X2/n2  > "3 

X2  ^ «1«3/Xl 
X,  < «2 

X,X2//l2  ^ ”3 
n^nJx^  < X2 


X,  < «2 
X,X2/n2  < «3 


Order 

First  Reduction 

Second  Reduction  ( 

A 

n^n^|x^ 

/Jl«2n3/x,X2 

B 

«l"3/Xl 

n^nJx^ 

D 

«2”3/X2 

n^n^rtifXiXi 

A 

n,n2/x, 

«i”2«3/AiX2 

B 

«l«3/Xl 

«2 

D 

ni'hlxi 

i A 

/i,n2/x, 

«1«2"3/AiX2 

B 

«1«3/Xl 

«I«2''3/AiX2 

! "" 

«2''3/A2 

«I«2'»3/a-iX2 

A 

«,W2/x, 

n,/i2/x, 

B 

«I«3/Ai 

n,/j3/x, 

, D 

«2 

niHilXi 

ni«2/x, 

nifijxi 

/7,/13/x, 

«2 

1 D 

«2 

«1«2/Xl 

A 

”1 

n,«2n3/x,X2 

B 

«l”3/Xl 

n,A/3/x, 

D 

”2"3/X2 

/J,/?2«3/x,X2 

A 

"1 

n^n^nJx^X2 

B 

n^n^jxy 

«2 

D 

n,rtj/J3/x,X2 

A 

"1 

«,rt2«3/»'l''2 

B 

W./tj/x, 

nifiinilXiXi 

D 

«2«3/A2 

n^n^nj/x^Xi 

A 

1 

1 w, 

B 

^ n, 

w, 

D 

^ n, 

"1 

1 


FIG.  1 CONFIDENCE  LEVEL  ACHIEVED  BY  EXACT  LIMIT  WHEN  n = 10, 


Legend 

X = Touch  point 
0,7287  value  of  p,  Pj 
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The  surface  touches  the  plane  all 
along  this  line,  p^  Pj  = 0,7943 


\ \ Pi  P2  = 0,5504  / 

\ \ X6071 

\ y\  : :-7287 

p,  p=0,4483'. 

\^457  V 

X4133  ^ ^ 


^439  \ 


•5504 


•4483 


•3542 


FIG.  5 POINTS  AT  WHICH  THE  SURFACE  OF  ACHIEVED  CONFIDENCE 
OF  THE  EXACT  LIMITS  OF  LIPOW  AND  RILEY’^  TOUCHES 
THE  PLANE  OF  CONFIDENCE  0,9. 


FIG.  7 CONFIDENCE  LEVELS  ACHIEVED  BY  THE  SR  METHOD  WHEN  n 
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FIG.  8 COMPARISON  OF  ACHIEVED  CONFIDENCE  LEVELS  OF  THE  MML  AND  SR  METHODS  WHEN  n 
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